Web Survey Bibliography
In interpreting questions, respondents extract meaning from how the information in a questionnaire is shaped, spaced, and shaded. We carried out five experiments using five questions measured on a five point scale to investigate how the effects of visual heuristics affect the answers to survey questions and test whether effective question writing (using numerical and verbal labels) may overrule visual effects. The experiments were fielded in the CentERpanel. This is an Internet-based panel, although there is no need to have a personal computer with an Internet connection. If necessary, equipment is provided by CentERdata (the agency). The recruitment of new panel members is done through a random sample of landline numbers of candidates. The panel is designed to be representative of the Dutch population. We demonstrate that respondents use the “middle means typical” heuristic only in a polar point scale. When numbers 1 to 5 are added to the polar point format and with fully labeled scales, the effect of spacing between response options disappears. In a second experiment we show that respondents are confused when options are presented in an inconsistent order and do not follow the “left and top means first” heuristic. In a third experiment we find that inter-item correlations are higher in a polar point scale when items are presented on a single screen instead of separate screens. The effect gets smaller when numbers 1 to 5 are added to the polar point scale. In a fully labeled scale no effect is found. In a fourth experiment, we found no evidence that respondents use the “up means good” heuristic: the use of a decremental or incremental scale did not result in different answer distributions. In a fifth experiment, we show that respondents use the “like means close” heuristic: the adding of numbers that differ both in sign and value (-2 to 2) and the adding of different shadings of red and green color affects respondents’ answers. The effect of different numbers was apparent in both a polar point and fully labeled format, while the effect of color was only apparent in the polar point format.
Conference Homepage (abstract)
Web survey bibliography - Dillman, D. A. (52)
- Improving the Effectiveness of Online Data Collection by Mixing Survey Modes; 2017; Dillman, D. A.; Hao, F.; Millar, M. M.
- On Climbing Stairs Many Steps at a Time: The New Normal in Survey Methodology; 2015; Dillman, D. A.
- Improving Response to Household Surveys Using Mail Contact to Request Responses over the Internet: Results...; 2015; Dillman, D. A.
- Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 4th Edition; 2014; Dillman, D. A., Smyth, J. D., Christian, L. M.
- Achieving Synergy Across Survey Modes: Mail Contact and Web Responses from Address-Based Samples; 2013; Dillman, D. A.
- An Experimental Evaluation of How Mode Sequence for offering Internet, Mail and Telephone Options Affects...; 2013; Finamore, J., Dillman, D. A.
- Attitudes of Nebraska Residents on Nebraska Water Management; 2013; Dillman, D. A., Edwards, M. L.
- Mode Effects in Mixed-Mode Surveys: Prevention, Diagnostics, and Adjustment 1; 2013; de Leeuw, E. D., Dillman, D. A., Schouten, B.
- Understanding the resilience of mail-back survey methods: An analysis of 20 years of change in response...; 2012; Rookey, B. D., Le, L., Littlejohn, M., Dillman, D. A.
- Intensifying the Request: Results from an Experiment on Improving Internet Response Rates for Address...; 2012; Messer, B. L., Dillman, D. A.
- Do Mail and Internet Surveys Produce Different Item Nonresponse Rates? An Experiment Using Random Mode...; 2012; Millar, M. M., Dillman, D. A.
- Determinants of Item Nonresponse to Web and Mail Respondents in Three Address-Based Mixed-Mode Surveys...; 2012; Messer, B. L., Edwards, M. L., Dillman, D. A.
- How Visual Design Affects the Interpretability of Survey Questions; 2011; Toepoel, V., Dillman, D. A.
- Surveying the General Public over the Internet Using Address-Based Sampling and Mail Contact Procedures...; 2011; Messer, B. L., Dillman, D. A.
- Creating Effective Designs for Mixed-Mode Surveys; 2011; Dillman, D. A.
- In the Face of Declining Budgets: The Student Experience at Washington State University ; 2011; Allen, T., Dillman, D. A., Garza, B., Millar, M. M.
- IVR and web administration in structured interviews utilizing rating scales: exploring the role of motivation...; 2011; Yang, Y., Callegaro, M., Bhola, D. S., Dillman, D. A.
- Research synthesis. AAPOR report on online panels; 2010; Brick, J. M., Baker, R., Blumberg, S. J., Couper, M. P., Courtright, M., Dennis, J. M., Dillman, D....
- Mixed-mode surveys; 2010; Dillman, D. A., Messer, B. L.
- Improving Response to Mail and Web Mixed-Mode Surveys: The Effects of Offering Choice on Survey Response...; 2010; Millar, M. M., Dillman, D. A.
- Questionnaire Design Guidelines for Establishment Surveys; 2010; Morrison, R. L., Dillman, D. A., Christian, L. M.
- Does Visual Appeal Matter? Effects of Web Survey Aesthetics on Survey Quality; 2010; Mahon-Haft, T., Dillman, D. A.
- Using the Internet to Survey Small Towns and Communities: Limitations and Possibilities in the Early...; 2010; Smyth, J. D., Dillman, D. A., Christian, L. M., O'Neill, A. C.
- Response format effects on measurement of employment; 2009; Thomas, R. K., Dillman, D. A., Smyth, J. D.
- Words, Numbers and Visual Heuristics in Web Surveys: Is There a Hierarchy of Importance?; 2009; Toepoel, V., Dillman, D. A.
- Improving the Effectiveness of Mail Contact Procedures to Obtain Survey Response Over the Internet for...; 2009; Messer, B. L., Dillman, D. A.
- The Difficult but Essential Challenge of Designing Mixed-Mode Surveys; 2009; Dillman, D. A.
- Using Mail Contact to Sample and Encourage Submission of Questionnaire Answers Over the Internet; 2009; Dillman, D. A., Messer, B. L., Millar, M. M.
- Visual Heuristics and Answer Formats in Rating Scales; 2009; Toepoel, V. Dillman, D. A.
- Open-Ended Questions in Web Surveys: Can Increasing the Size of Answer Boxes and Providing Extra Verbal...; 2009; Smyth, J. D., Dillman, D. A., Christian, L. M., McBride, M.
- Are Mode Preferences Real?; 2009; Millar, M. M., O'Neill, A. C., Dillman, D. A.
- Designing Scalar Questions for Web Surveys; 2009; Christian, L. M., Parsons, N. L., Dillman, D. A.
- Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice...; 2009; Dillman, D. A., Phelps, G., Tortora, R. D., Swift, K., Kohrell, J., Berck, J., Messer, B. L.
- Some consequences of survey mode changes in longitudinal surveys; 2008; Dillman, D. A.
- To Mix or Not To Mix Survey Modes; 2008; Dillman, D. A.
- ...; 2008; Dillman, D. A., Smyth, J. D., Christian, L. M., O'Neill, A. C.
- Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method ; 2008; Dillman, D. A., Christian, L. M., Smyth, J. D.
- How can we convince the general public to respond to internet surveys?; 2008; Dillman, D. A.
- Combining Mail and Internet Methods to Conduct Household Surveys of the General Public: A New Methodology...; 2008; Dillman, D. A., Smyth, J. D., Christian, L. M., Oneill, A.
- Does the Inclusion of Mail and Web Alternatives in a Probability-Based Household Panel Improve the Accuracy...; 2008; Rookey, B. D., Dillman, D. A., Hanway, S.
- Does a Probability-Based Household Panel Benefit from Assignment to Postal Response as an Alternative...; 2008; Rookey, B. D., Hanway, S., Dillman, D. A.
- Context effects in Internet Surveys: New issues and evidence; 2007; Smyth, J. D., Dillman, D. A., Christian, L. M.
- Guidelines for Designing Questionnaires for Administration in Different Modes; 2007; Martin, E., Hunter, J. E., DeMaio, T., Hill, Jo., Reiser, C., Gerber, E., Styles, K., Dillman, D. A.
- Is it Possible to Obtain Equivalent Answers to Scalar Questions in Web and Telephone Surveys?; 2006; Christian, L. M., Dillman, D. A., Smyth, J. D.
- Open-Ended Questions in Web and Telephone Surveys; 2006; Smyth, J. D., Dillman, D. A., Christian, L. M., McBride, M.
- Comparing Check-All and Forced-Choice Question Formats in Web Surveys: The Role of Satisficing, Depth...; 2005; Smyth, J. D., Dillman, D. A., Christian, L. M., Stern, M. J.
- Response latency as an indicator of optimizing. A study comparing job applicants and job incumbents...; 2004; Callegaro, M., Yang, Y., Bhola, D. S., Dillman, D. A.
- The Influence of Graphical and Symbolic Language Manipulations on Responses to Self-Administered Questions...; 2004; Christian, L. M., Dillman, D. A.
- Results from cognitive interviews of nsf earned doctorate Web survey; 2002; Altheimer, I., Dillman, D. A.
- Survey Nonresponse; 2001; Groves, R. M., Dillman, D. A., Eltinge, J. L.